What do generalized entropies look like? An axiomatic approach for complex, 

non-ergodic systems 
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Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must 
be of Boltzmann-Gibbs type, S = — Pi log Pi- Here we note that in physical systems one of these 
axioms may be violated. For non-ergodic systems the so called separation axiom (Shannon-Khinchin 
axiom 4) will in general not be valid. We show that when this axiom is violated the entropy takes 
a more general form, Sc,d oc r(d +1,1 — clogpi), where c and d are scaling exponents and 
r{a,b) is the incomplete gamma function. The exponents (c, d) define equivalence classes for all 
interacting and non interacting systems and unambiguously characterize any statistical system in 
its thermodynamic limit. The proof is possible because of two newly discovered scaling laws which 
any entropic form has to fulfill, if the first three Shannon-Khinchin axioms hold, (c, d) can be used to 
define equivalence classes of statistical systems. A series of known entropies can be classified in terms 
of these equivalence classes. We show that the corresponding distribution functions are special forms 
of Lambert-W exponentials containing - as special cases - Boltzmann, stretched exponential and 
Tsallis distributions (power-laws). In the derivation we assume trace form entropies, S = '^^gipi), 
with g some function, however more general entropic forms can be classified along the same scaling 
analysis. In this contribution we largely follow the lines of thought presented in [l|. 

PACS numbers: 05.20.-y, 02.50.Cw, 05.90.-|-m 



I. INTRODUCTION 

Theorem number 2 in the seminal 1948 paper, The 
Mathematical Theory of Communication [2], by Claude 
Shannon, proves the existence of the one and only form of 
entropy, given that three fundamental requirements hold. 
A few years later A.L Khinchin remarked in his Mathe- 
matical Foundations of Information Theory Q: "How- 
ever, Shannon's treatment is not always sufficiently com- 
plete and mathematically correct so that, besides hav- 
ing to free the theory from practical details, in many in- 
stances I have amplified and changed both the statement 
of definitions and the statement of proofs of theorems." 
Khinchin adds a fourth axiom. The three fundamental 
requirements of Shannon, in the 'amplified' version of 
Khinchin, are known as the Shannon-Khinchin (SK) ax- 
ioms. These axioms list the requirements needed for an 
entropy to be a reasonable measure of the 'uncertainty' 
about a finite probabilistic system. Khinchin further sug- 
gests to also use entropy as a measure of the information 
gained about a system when making an 'experiment', i.e. 
by observing a realization of the probabilistic system. 

Khinchin's first axiom states that for a system with 
W potential outcomes (states) each of which is given by 
a probability Pi > 0, with Y^^iPi — 1; the entropy 
S{pi,--- ,pw) as a measure of uncertainty about the 
system must take its maximum for the equi-distribution 
Pi = l/W, for ah i. 

Khinchin's second axiom (missing in ^]) states that 
any entropy should remain invariant under adding zero- 
probability states to the system, i.e. S{pi,--- ,pw) — 

S{pi,--- ,Pw,0)- 

Khinchin's third axiom (separability axiom) finally 
makes a statement of the composition of two finite proba- 



bilistic systems A and B. If the systems are independent 
of each other, entropy should be additive, meaning that 
the entropy of the combined system A + B should be the 
sum of the individual systems, S{A + B) = S{A) + S{B). 
If the two systems are dependent on each other, the en- 
tropy of the combined system, i.e. the information given 
by the realization of the two finite schemes A and B, 
S{A -\- B), is equal to the information gained by a re- 
alization of system A, S{A), plus the mathematical ex- 
pectation of information gained by a realization of sys- 
tem B, after the realization of system A, S{A + B) = 
SiA) + S\AiB). 

Khinchin's fourth axiom is the requirement that en- 
tropy is a continuous function of all its arguments pi and 
does not depend on anything else. 

Given these axioms, the Uniqueness theorem Q 
states that the one and only possible entropy is 
S{pi,--- ,pw) — —k^^iPilogpi, where k is an arbi- 
trary positive constant. The result is of course the same 
as Shannon's. We call the combination of 4 axioms the 
Shannon-Khinchin (SK) axioms. 

From information theory now to physics, where sys- 
tems may exist that violate the separability axiom. This 
might especially be the case for non-ergodic, complex sys- 
tems exhibiting long-range and strong interactions. Such 
complex systems may show extremely rich behavior in 
contrast to simple ones, such as gases. There exists some 
hope that it should be possible to understand such sys- 
tems also on a thermodynamical basis, meaning that a 
few measurable quantities would be sufficient to under- 
stand their macroscopic phenomena. If this would be 
possible, through an equivalent to the second law of ther- 
modynamics, some appropriate entropy would enter as a 
fundamental concept relating the number of microstates 
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in the system to its macroscopic properties. Guided by 
this hope, a series of so called generalized entropies have 
been suggested over the past decades, see 0-0 and Ta- 
ble 1. These entropies have been designed for different 
purposes and have not been related to a fundamental ori- 
gin. Here we ask how generalized entropies can look like 
if they fulfill some of the Shannon-Khinchin axioms, but 
explicitly violate the separability axiom. We do this ax- 
iomatically as first presented in By doing so we can 
relate a large class of generalized entropies to a single 
root. 

The reason why this axiom is violated in some physi- 
cal, biological or social systems is broken ergodicity, i.e. 
that not all regions in phase space are visited and many 
micro states are effectively 'forbidden'. Entropy relates 
the number of micro states of a system to an extensive 
quantity, which plays the fundamental role in the systems 
thermodynamical description. Extensive means that if 
two initially isolated, i.e. sufficiently separated systems, 
A and B, with Wa and Wb the respective numbers of 
states, are brought together, the entropy of the combined 
system A + B is SiWA+s) = SiWA) + SiWu)- Wa+b 
is the number of states in the combined system A + B. 
This is not to be confused with additivity which is the 
property that S{WaWb) = S{Wa) + S{Wb). Both, ex- 
tensivity and additivity coincide if number of states in the 
combined system is Wa+b = WaWb- Clearly, for a non- 
interacting system Boltzmann-Gibbs-Shannon entropy, 

'S'bgH — ~ X^r^ Pi l^Pij extensive and additive. By 
'non-interacting' (short-range, ergodic, sufficiently mix- 
ing, Markovian, ...) systems we mean Wa+b — WaWb- 
For interacting statistical systems the latter is in gen- 
eral not true; phase space is only partly visited and 
Wa+b < WaWb- In this case, an additive entropy such 
as Boltzmann-Gibbs can no longer be extensive and vice 
versa. To ensure extensivity of entropy, an entropic form 
should be found for the particular interacting statistical 
systems at hand. These entropic forms are called qener- 
alized entropies and usually assume trace form [3-01 



for the equi-distribution Pi = 1/W (for all i) implies 
that g is a concave function. 

• SK3: The requirement that adding a zero- 
probability state to a system, W-\-l withpvv+i = 0, 
does not change the entropy, implies that ^(0) = 0. 

• SK4 (separability axiom) : The entropy of a system 
- composed of sub-systems A and B - equals the 
entropy of A plus the expectation value of the en- 
tropy of B, conditional on A. Note that this also 
corresponds exactly to Markovian processes. 

As mentioned, if SKI to SK4 hold, the only possible en- 
tropy is the Boltzmann-Gibbs-Shannon entropy. We are 
now going to derive the extensive entropy when the sep- 
arability axiom SK4 is violated. Obviously this entropy 
will be more general and should contain BG entropy as 
a special case. 

We now assume that axioms SKI, SK2, SK3 hold, i.e. 
we restrict ourselves to trace form entropies with g con- 
tinuous, concave and g(0) — 0. These systems we call 
admissible systems. 

This generalized entropy for (large) admissible statis- 
tical systems (SK1-SK3 hold) is derived from two hith- 
erto unexplored fundamental scaling laws of extensive 
entropies. Both scaling laws are characterized by ex- 
ponents c and d, respectively, which allow to uniquely 
define equivalence classes of entropies, meaning that two 
entropies are equivalent in the thermodynamic limit if 
their exponents (c, d) coincide. Each admissible system 
belongs to one of these equivalence classes (c, d) , [l| . 

In terms of the exponents (c, d) we show in the 
following that all generalized entropies have the form 
Sc4 « Y.Y^id + 1,1 - clogp,), with r(a,6) = 
dt t"'^^ exp{—t) the incomplete Gamma-function. 

Admissible systems when combined with a maximum 
entropy principle show remarkably simple mathematical 
properties as recently demonstrated in [lo| . 
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W being the number of states. Obviously not all general- 
ized entropic forms are of this type. Renyi entropy e.g. is 
of the form G{^Y diPi))^ with G a monotonic function. 
We use trace forms Eq. ([1]) for simplicity. Renyi forms 
can be studied in exactly the same way as will be shown, 
however at more technical cost. 

Let us revisit the Shannon-Khinchin axioms in the light 
of generalized entropies of trace form Eq. ([T]). Specif- 
ically axioms SK1-SK3 (now re-ordered) have implica- 
tions on the functional form of g 

• SKI: The requirement that S* depends continuously 
on p implies that 5 is a continuous function. 

• SK2: The requirement that the entropy is maximal 



II. THE SCALING LAWS OF ENTROPIES 

We discuss two - a primary and a secondary - scal- 
ing properties of generalized entropies of trace form and 
assume the validity of the first 3 KS axioms. For equi- 
distribution Pi = (for all i) obviously, "^^i g{Pi) = 
Wgi^). 

The primary scaling law is found from the relation 
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We define a scaling function 



f{z) = lini 



g{zx) 



g{x) 



(0 < z < 1) 



(2) 



(3) 



This function / for systems satisfying SK1-SK3, but 
violating SK4, can only be a power f{z) = z^, with 
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where A 



cdr 



and r is an arbitrary constant r > 
r the function gc,d,r is a 



l-(l-c)r 

(see below). For all choices of 
representative of the class (c, d). This allows to choose 
r as a suitable function of c and d. For example choose 
r = (1 — c + cd)~^, so that A= 1, and 



eErr(l + d, l-clnp,) 



1 



cc? 



1 



cd 



(9) 

It can be easily verified that Sc.d has the correct asymp- 
totic properties, see Theorem 3 in the appendix. 



FIG. 1: Equivalence classes of entropies parametrized in the 
(c, d)-plane, with their associated distribution functions. BG 
entropy corresponds to (1, 1), Tsallis entropy to (c, 0), and en- 
tropies for stretched exponentials to (l,d > 0). All entropies 
leading to distribution functions with compact support, be- 
long to equivalence class (1, 0). An example are Sq entropies 
with q > 1 (using the maximum entropy principle with usual 
expectation values in the constraints [ol, lla|). 



A. Special cases of entropic equivalence classes 



Let us look at some specific equivalence classes (c, d) 

• Boltzmann-Gibbs entropy belongs to the (c, d) — 
(1, 1) class. One gets from Eq. ^ 



(10) 



< c < 1, given / being continuous. This mathematical 
fact is shown as a theorem in the appendix (Theorem 1). 
Inserting Eq. ^ in Eq. ^ gives the primary asymptotic 
scaling law for entropies 



hm ^ . A-^ 



W^OO Sg{W) 



(4) 



To identify the secondary scaling law start from the 
observation that 



hm ^A-^ ^ 1 



(5) 



W^oo S{W) 

and substitute A in Eq. ([5]) by A W°-. We define hc{a) 



hc(a) = lim ' W 

^ ' W^oo S{W) 



lim , ( , (6) 



with X = 1/W. It can be proved as a mathematical fact 
(Theorem 2 in the appendix) that 



hc{a) = (1 + a)*^ {d constant) 



(7) 



Remarkably, he does not explicitly depend on c. hc{a) is 
an asymptotic property which is independent of property 
Eq. dl]). Note that if c = 1, concavity of g implies d > 0. 
In principle this scheme can be iterated to higher orders, 
see appendix. 



III. DERIVATION OF ENTROPY 

We now ask which families of entropies, i.e. functions 
gc.d fulfill the primary and the secondary scaling law. A 
particularly simple choice is 



gc4,r{x) = rA~'^e^ T {I + d , A - clnx) 



(8) 



• Tsallis entropy belongs to the (c, d) = (c, 0) class. 
From Eq. ([5]) and the choice r = 1/(1 — c) (see 
below) we get 



ScAp] = Y.t9cfi{Pi) 



c-l 



1 



(11) 



Note, that although the pointwise limit c — > 1 
of Tsallis entropy yields BG entropy, the asymp- 
totic properties (c, 0) do not change continuously 
to (1, 1) in this limit! In other words the thermo- 
dynamic limit and the limit c — 1 do not commute. 

• The entropy related to stretched exponentials Q 
belongs to the {c,d) = {l,d) classes, see Table 1. 
As a specific example we compute the (c, d) = {1, 2) 
case, 

5i,2W = 2 ^l-^panp,j +i^p,(lnpO', (12) 

leading to a superposition of two entropy terms, 
the asymptotic behavior being dominated by the 
second. 

• All entropies associated to distributions with com- 
pact support belong to (c, d) = (1, 0). Clearly, all of 
these have the same (trivial) asymptotic behavior. 

Other entropies which are special cases of our scheme are 
found in Table 1. 

Inversely, for any given entropy we are now in the 
remarkable position to characterize all large SK1-SK3 
systems by a pair of two exponents (c, d) , i.e. their 
scaling functions / and he- See Fig. [T] For exam- 
ple, for gBcix) — — a;ln(a;) we have f{z) = z, i.e. 
c — 1, and hc{a) = 1 + a, i.e. d = 1. Sbg there- 
fore belongs to the universality class (c, d) — (1, 1). For 
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FIG. 2: Distribution functions based on the Lambert ex- 
ponential, p{x) — £c.d,r{—x), for various (c, d) values, (a) 
Stretched exponential limit c — >■ 1. It includes the Boltzmann 
distribution for d — 1. (b) d — >■ limit - i.e. the g-exponential 
limit. In (a) and (b) the black dashed lines represent the 
stretched exponential (c = 1) or g-exponential (d — 0) limit 
functions, (c) General case for distribution functions for sev- 
eral values of (c, d) different from the limits c ~ 1 or d ~ 0. 
They must not be confused with power-laws. 



gq{x) — [x — x'^)/{l — q) (Tsallis entropy) and < q < 1 
one finds f{z) — z'?, i.e. c — q and hc{a) = 1, i.e. d = 0, 
and Tsallis entropy, Sq, belongs to the universality class 
(c, d) = {q, 0). Other examples are listed in Table 1. 

The universality classes {c,d) are equivalence classes 
with the equivalence relation given by: Qa = Ca — 

cp and da — dp. This relation partitions the space of all 
admissible g into equivalence classes completely specified 
by the pair (c, d). 



IV. DISTRIBUTION FUNCTIONS 



alized logarithm A is found in closed form 
c-i 1 - (1 - c)r 



J^c.dA^) = r - rx 



1 d 



I 



In; 



(13) 



and its inverse function is 



with the constant B = j^^^^^^^-^J^^ exp 



(14) 
The 



Vl-(l-c)'-, 

function Wfc is the fc'th branch of the Lambert-W 
function which - as a solution to the equation x = 
W(a:) exp(>V(a;)) - has only two real solutions Wk, the 
branch fc = and branch fc = — 1. Branch fc = covers 
the classes for d > 0, branch k — —I those for d < 0. 



A. Special cases of distribution functions 

It is easy to verify that the class (c, d) = (1, 1) leads 
to Boltzmann distributions, and the class (c, d) — (c, 0) 
yields power-laws, or more precisely, Tsallis distribu- 
tions i.e. g-exponentials. All classes associated with 
(c, d) = (l,(i), for d > are associated with stretched 
exponential distributions. Expanding the fc = branch 
of the Lambert-W function Wq {x) ~ x — x"^ + . . . for 
1 3> I a; I, the limit c — >■ 1 is seen to be a stretched expo- 
nential 



lim £c 



-dr|^(l-f ) d _i| 



(15) 



Clearly, r does not effect its asymptotic properties (tail 
of the distribution), but can be used to incorporate finite 
size properties of the distribution function for small x. 
Examples of all of the above distribution functions are 
shown in Fig. [5] 



B. Finite size effects and the parameter r 

In Eq. © we chose r = (1 — c 4- cd)~^ . This is not 
the most general case and the only limitations on r - if 
one requires the generalized logarithms to have the usual 
properties A(l) = and A'(l) = 1 to hold - are 



d > 
d = 
d < 



r < 
r = 
r > 



1 

l-c 

1 

l-c 

1 

l-c 



(16) 



Distribution functions associated with our F-entropy, 
Eq. ©, can be derived from so-called generalized log- 
arithms of the entropy. Under the maximum entropy 
principle (given ordinary constraints) the inverse func- 
tions of these logarithms, £ = A~^, are the distribution 
functions, p(e) = fc,d,r(— e)- Following [1, the gener- 



Every choice of r gives a representative of the equivalence 
class (c, d), i.e. r has no effect on the thermodynamic 
limit and therefore can be used to encode potential finite- 
size characteristics of a system at hand. In case of no such 
effects, practical choices are r = (1 — c + cd)~^ for d > 0, 
and r = exp(— d)/(l — c) for d < 0. 
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TABLE I: Several trace form entropies S — gipi), for which SK1-SK3 hold. They are special cases of the entropy given 
in Eq. ^ and their asymptotic behavior is uniquely determined by c and d. It can be seen immediately that Sq>i, Sb and Se 
are asymptotically identical; so are S'q<i and Sk., as well as Sr, and S-y. 
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5,>i(p) = ^^ (g>l) 
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S.(p) = E,r(2±i,-InpO-p.r(2±i) (r;>0) 


1 




[5] 


^^(P) = E,Pani/^(lM) 
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d= 1/7 


[11], footnote 11, page 60 


S;9(p) = E.pfln(l/pO 


c = /3 


1 


fl2] 



A PHYSICAL SYSTEM AS AN EXAMPLE 



VI. RENYI-TYPE ENTROPIES 



From axiomatic derivations no physical relevance can 
be inferred. To demonstrate the applicability of the pro- 
posed classification scheme to concrete physical systems, 
consider over-damped interacting particles moving in a 
narrow channel. 



[iVi = 2\j{ri - Tj) + F{ri) 4- r\(ri,t) 



(17) 



where Vi is velocity of ith particle, fi viscosity, F an ex- 
ternal force, J{f) a linear repulsive particle-particle in- 
teraction, ry uncorrelated thermal noise with (?/) = and 
(rj'^) = with A a characteristic length of the pair 
interaction. For details see [l^l, where the non-linear 
Fokker-Planck equation for the spatial distribution p(x) 
of the particles was solved. It was shown that the cor- 
responding entropy is a superposition of BG and Tsallis 
entropy with q — 2. Note the similarity to our exam- 
ple in Eq. (jl2p where a similar superposition emerges 
naturally. At high temperatures the BG contribution 
dominates i.e. particles diffuse normally, while at zero 
temperature the system is governed by Tsallis entropy 
with a non- Gaussian parabolic diffusion profile of com- 
pact support. The concrete stationary distribution func- 
tions given in [22j can be used to demonstrate that the 
system is found either in the asymptotic equivalence class 
(c, d) = (1, 1) (BG entropy) or in (c, d) = (1, 0) (compact 
support entropies) depending on the temperature of the 
heat bath. Expressing the integral by a discrete sum the 
entropy given in (2^ can be used to demonstrate the ex- 
istence of the two sets of classes. This shows that the 
classification is applicable for concrete physical systems 
and suggests further that the equivalence class (c, d) may 
even depend on macro variables of the system such as the 
temperature of the heat bath. 



Renyi entropy is obtained by relaxing SK4 to the pure 
(unconditional) additivity condition. Following the same 
scaling idea for Renyi-type entropies, S — G{J2i=i g{Pi))i 
with G and g some functions, one gets 



lim 



S{\W) 
S{W) 



= lim 



G{s) 



(18) 



where fg{z) = lim,j:^o g{zx)/ g{x). The expression 
fds) = linis G{sy)/G{s), provides the starting point for 
deeper analysis which now gets more involved. In par- 
ticular, for Renyi entropy with G{x) = ln(a;)/(l — a) 
and g{x) = x" , the asymptotic properties yield the class 
(c, d) — (1, 1), (BG entropy) meaning that Renyi entropy 
is additive. However, in contrast to the trace form en- 
tropies used above, Renyi entropy can be shown to be 
not Lesche stable, as was observed before [l6l - l20l | . All of 

the S = X)!^ 9{Pi) entropies can be shown to be Lesche 
stable, see Theorem 4 in the appendix. 



VII. DISCUSSION 

We studied the scaling laws of trace form entropies 
which are constrained by the first three Shannon- 
Khinchin axioms in the thermodynamic limit. In analogy 
to critical exponents these laws are characterized by two 
scaling exponents (c, d), which define equivalence rela- 
tions on entropic forms. We showed that a single entropic 
form - parametrized by the two exponents - covers all ad- 
missible systems (Shannon-Khinchin axioms 1-3 hold, 4 
is violated). In other words every statistical system has 
its pair of unique exponents in the large size limit, its en- 
tropy is then given by Sc,d ^ T (1 -I- d , 1 — clnp^). 

The exponents for BG systems are {c,d) = (1,1), 
systems characterized by stretched exponentials belong 
to the class (c, d) = (l,(i), and Tsallis systems have 
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(c, d) = (9,0). In the context of a maximum entropy 
principle, the associated distribution functions of all sys- 
tems (c, d) are shown to belong to a class of exponentials 
involving Lambert- W functions, given in Eq. (jl4p . There 
are no other options for tails in distribution functions 
other than these. 

The equivalence classes characterized by the exponents 
(c, d) form basins of asymptotic equivalence. In gen- 
eral these basins characterize interacting statistical (non- 
additive) systems. There exists an analogy between these 
basins of asymptotic equivalence and the basin of attrac- 
tion of weakly interacting, uncorrelated systems subject 
to the law of large numbers, i.e. the central limit theo- 
rem. Any system within a given equivalence class may 
show individual characteristics as long as it is small. Sys- 
tems belonging to the same class will start behaving sim- 
ilarly as they become larger and in the thermodynamic 
limit they become identical. Distribution functions con- 
verge to those uniquely determined by (c, d). 

Our framework shows that for non-interacting systems 



c = 1. Setting X = Wb in Eq. ^ and Eq. imme- 
diately implies S{WaWb)/S{Wa) ~ W^^". This means 
that if for such a system it would be true that c 7^ 1, then 
adding only a few independent states to a system would 
explosively change its entropy and extensivity would be 
strongly violated. A further interesting feature of admis- 
sible systems is that they all are what has been called 
Lesche stable. As a practical note Lesche stability corre- 
sponds one-to-one to the continuity of the scaling func- 
tion / and can now be checked by a trivial verification of 
this property (Eq. The proof is given in Theorem 

4 in the appendix. 

Finally we remark that the classification scheme for 
generalized entropic forms of type S — J2i 9iPi) '^^^ 
be extended to entropies of e.g. Renyi type, i.e. S = 
G{J2i9{Pi))- We demonstrated that generalized en- 
tropies can be applied to actual physical systems. We 
hypothesize that many complex statistical systems are 
indeed admissible systems of equivalence classes (c, d), 
with < c < 1. 
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VIII. APPENDIX 

Theorem 1: Let 5 be a continuous, concave function 
on [0, 1] with g{0) = and let /(z) — lim2.^o+ 9{zx)/g{x) 
be continuous, then / is of the form f{z) = with c G 
(0,1]. 

Proof. Note that f{ab) = linix-i-o 9{obx) / g{x) ~ 
\iu,,^o{g{abx)/g{bx)){g{bx)/g{x)) = f{a)f{b).^ Ah 
pathological solutions are excluded by the requirement 
that / is continuous. So f{ab) = f(a)f{b) implies that 
f{z) = z'^ is the only possible solution of this equation. 
Further, since gifS) = 0, also Xivcix^q gi^x^ j g{x) = 0, and 
it follows that /(O) — 0. This necessarily implies that 
c > 0. f{z) = also has to be concave since g{zx)/g{x) 
is concave in z for arbitrarily small, fixed a; > 0. There- 
fore c < 1. □ 

Theorem 2: Let g be like in Theorem 1 and let f{z) = 
z'^ then he in Eq. (jG]) is a constant of the form hc{a) — 
{1 + aY for some constant d. 

Proof. We can determine hc{a) again by a similar trick 
as we have used for /. 

h,{a) =lim,^of^^ 

^h,X^-\)KXb-\) , 

for some constant 6. By a simple transformation of vari- 
ables, a = bb' — l, one gets hc{bb' — l) = hc{b—l)hc{b' — I). 
Setting H{x) ~ hc{x — 1) one again gets H{bb') = 
H{b)H{b'). So H{x) = x'^ for some constant d and con- 
sequently hc{a) is of the form (1 -I- a)**. □ 
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Theorem 3: The entropy based on gc.d,r, Eq. ([8]), has 
the deshed asymptotic properties. 

Proof. Let g be hke in Theorem 1, i.e. let f{z) = z'^ with 
< c < 1, then 



hm , = c . 

:i;^0+ ^g{x) 



(19) 



Consider 



lim,. 



g(aj)-g(^a:) 



1 [ g(a^)-g(za-0 _ 

1-z I 9(2;) y 2-1 



Taking the limit z ^> 1 on both sides completes 
the first part of the proof. Further, two func- 
tions gA and gs generate equivalent entropic forms if 
lim2._j.o+ 9a{x)/ gsix) = (j) and < < 00. This clearly 
is true since 



1™ , 9Aizx) 



_ gAjzx) gsix) gsizx) 
gsizx) qaM gnix) 

9b(x) 
— lim , 9b(zx) 



By an analogous argument the same result can be ob- 
tained for the second asymptotic property, Eq. A 
simple lemma is that given 5b (x) = agA{bx), for some 
suitable constants a and b, then gs and gA are equivalent. 
A lemma, following from Eq. is that 



lim 



gAix) 



= lim 



g'Ai^) 



->o+ gsix) x^o+ g'g{x) ' 

which is just the rule of L'Hospital shown to hold for 
the considered families of functions g. This is true since, 
either \\va.x^Q+ gA{x) / gsix) — (f> with < < 00 and 
ca = cb, i.e. gA and gs are equivalent, or gA and gB are 
inequivalent, i.e. ca 7^ cb but <p — oi ip ^ 00. 

So if one can find a function gtost, having the de- 
sired asymptotic exponents c and d, it suffices to show 
that < -lim^^o+ ^c.dA^) /d'tcstix) < 00, where Ac^.r 
is the generalized logarithm Eq. associated with 

the generalized entropy Eq. ©. The test function 
5tcst(a;) = a;'^log(l/a;)'^ is of class (c, d), as can be verified 
easily. Unfortunately f/tcst can not be used to define the 
generalized entropy due to several technicalities. In par- 
ticular gtest lacks concavity around x ^ 1 for a consider- 
able range of (c, d) values, which then makes it impossible 
to define proper generalized logarithms and generalized 
exponential functions on the entire interval x G [0,1]. 
However, we only need the asymptotic properties of gtest 
and for x ~ the function gt^st does not violate concav- 
ity or any other required condition. The first derivative 
is gi,,,{x) = X--' log(l/a;)'^-i(clog(l/a:) - d). With this 
we finally get 



lim 



A.,dA^) _r-D-^{^y-' [log(f)]' 



0+ 5Lst(2^) a;--ilog(i)''-i(clogi-d) 



cDd^ ■ 
(20) 



1 — c 

Since < < 00 this proves that the Gamma-entropy 
gc,d,r, Eq. (10), represents the equivalence classes {c,d). 

□ 

Lesche stability of trace form entropies 

The Lesche stability criterion is a uniform-equi- 
continuity property of functionals S[p] on families of 
probability functions {p^^^}'w=i where p^^^ = {pj^jj^i. 
The criterion is phrased as follows: 

Let p^^^f and q'^^^ be probabilities on W states. An 
entropic form S is Lesche stable if for all e > and all 
W there is a <5 > such that 



-5[(7(^)]| < eS{W) 



(21) 

where S{W) is again the maximal possible entropy for 
W states. 

We characterize Lesche stability on the class of our 
generalized entropic forms in terms of continuity of / in 

Theorem 4: Let pi > be a probability and W 
the number of states i. Let 5 be a concave, continu- 
ous function on [0, 1], continuously differentiable on the 
semi-open interval (0, 1] and g(0) — 0. The entropic 
form Sg[p] — giPi) is Lesche stable iff the function 

f{z) — lim2;_j.o g{zx) / g{x) is continuous on z £ [0, 1]. 

Proof. SK2 states that maximal entropy is given by 
Sg{W) — Wg{l/W). We identify the worst case scenario 
for \Sg\p\ — >S'c,[g]|, where p and q are probabilities on the 
W states. For this maximize G[p,q] = \Sg[p] — Sg[q]\ — 

"(Ej -Pi - 1) - ^(Ej * - 1) - 7(E JP» - 9d - ^) > where a, j3 
and 7 are Lagrange multipliers. Without loss of general- 
ity assume that Sg [p] > Sg [q\ . Thus condition dG / dpi = 
gives g'{pi) -\- 7sign(pi — qi) — a = 0, where g' is the 
derivative of g and sign is the sign function. Similarly, 
dG/dqi = leads to g' {qi) + ^ sign{pi — qi) + [3 = 0. From 
this we see that both p and q can only possess two values 
p+, p_ and q+ and q-, where one can assume (without 
loss of generality) that p+ > g_ and q+ > p_. We can 
now assume that for w indices i p+ — pi > qi = q^ and 
for W — w indices j p- = pj < qj = q+ where w may 
range from 1 to — 1. This leads to seven equations 



wp+ + (W — w)p- = 1 , g'{p+) + 7 — a = 

wq- + (W — w)q+ = 1 , g'{p-) — 7 — a = 

w{p+-q-)- , g'(p+)-f 7 + /3 = 

^iW-w){p--q+) = 6 , 5'(p+)-7 + /3 = 



(22) 



which allow to express p-, q-, and q^ in terms of 

1 — wp^ 



P- 



q- 



W-w 
S 

1 — wp+ 5 
W-w ^ 2{W -w) 



(23) 
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Further we get the equation 

9\v+)~9'ip-)+9'{q+)-9'{q-) = Q ■ (24) 

Since g is concave g' is monotonically decreasing and 
9'{P+) ~ 9'{q-) > and g'{q+) - g'{p.) > 0. Thus Eq. 
(j24p has no solution, meaning that there is no extremum 
with p± and q± in (0, 1), and extrema are at the bound- 
aries. The possibihties arep+ = 1 orp_ = 0, then g+ = 1 
and ~ 0. Only = 1 or p_ = are compatible with 
the assumption that S[p] > S[q]; p+ = 1 is only a spe- 
cial case of p_ =0 with n = 1. Since g(0) = this 
immediately leads to the inequality 



\Sg[p] - Sg 



< (1 



9 \^ 2(1-0) IV 

^' 9iw) 
\<pw J 9 ycf,w 



9iw) 9iw) 



(25) 



yS > 0. This again means that either f{z) is discontin- 
uous at z = 1/0, or liuiz-^o f{z) > 0. Since g(0) — 
implies that /(O) — 0, f{z) has to be discontinuous at 
z = 0. □ 



Remark on higher order scaling exponents: 

In principle the scheme of finding scaling exponents 
can be iterated to higher orders, i.e. to find a se- 
quence of exponents (c, d, (i2, ds, . . . ). To see this define 
log("+^^(x) = log(log("^(x)), with log^^Hx) = log(x) and 



exp^"+^)(x) = exp(exp^"^(a;)), with exp^^^(x) = exp(a;) 
then up to a precision M 



M , 

(x) ^ n log^"^ - 



m— 1 



(28) 



where </> = w/VK is chosen such that the right hand side of 
the equation is maximal. Obviously, for any finite W the 
right hand side can always be made as small as needed 
by choosing (5 > small enough. Now take the limit 
W ^ oo. If / is continuous 



\S9\v\ - Sg\q\\ 



< 



< (1 

< (1 

< (5" 



2(1 - 0) 
l-(l-c^ 



1 - 



1-1 



(26) 



Lesche-stability of Sg follows since the right hand side of 
Eq. (|26p can be made smaller than any given e > by 
choosing S > small enough. This completes the first 
direction of the proof. 

If, on the other hand, Sg is not Lesche-stable then there 



exists an e > 0, such that \Sg[p] 
implying 



SM\/Sn 



(1 - <^)/ 



<5 



2(1 




ViV, 



(27) 



for small x. The n'th scaling exponent d„ can be deter- 
mined by 



lim 



g {Xnix)x) 



»-i / lot 



(m) 



n 



{xxl(x)) 



.^oA„(x)-5(x) Aj-^ \^ log(™)(i) 
where the generalized scaling factor reads 

A„(x)-i =xexp(") (^(l+a„)log(") (^^ 



(29) 



(30) 



Note that this is a nested scheme of sequentially in- 
corporating more information with higher levels, i.e. if 
g e {c,d,d2,...,d„+i), then also g G (c, d, ^2, ■ ■ • , 5n)- 
Using higher order asymptotic exponents makes it diffi- 
cult to find a concave representative gc,d.,d2,...,dM (^) ^i^'i 9 
quickly becomes experimentally inaccessible. For macro- 
scopic systems there may exist an upper limit M such 
that measuring d,„ for to > M becomes senseless. For 
example, consider a system with N ^ 10^^ particles and 
f2 oc states. Then x oc l/fl and it does not make any 
sense to measure or higher because log'^^^x) ^ —1.13 
is already negative, and log^^-* (x) is no longer defined on 
the principal branch of the logarithm. 




10^ 



10 
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(b) d=0.025, r=0.9/(1-c) 






c=0.2 




c=0.6 
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c=0.8 
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10^ 



10^ 



10 



-20 




■^(0.3, 4) 
■^(0.3-4) 
(0.7, 4) 
o (0.7-4) 



